You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "kevinjqliu (via GitHub)" <gi...@apache.org> on 2026/04/22 21:46:24 UTC

[PR] ci: add all `runtime-deps.txt` files [iceberg]

kevinjqliu opened a new pull request, #16081:
URL: https://github.com/apache/iceberg/pull/16081

   Context: #16080
   
   I checked out `main`, git pull and ran 
   ```
   ./gradlew generateRuntimeDeps -DallModules --rerun-tasks
   ```
   
   Committed the resulting files in this PR.
   
   `spark/v4.1/spark-runtime/runtime-deps.txt` was added in #16080


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134166898


##########
aws-bundle/runtime-deps.txt:
##########
@@ -0,0 +1,70 @@
+com.github.ben-manes.caffeine:caffeine:2.9.3

Review Comment:
   Note to other reviewers: The aws-bundle/LICENSE file and azure-bundle/LICENSE file both include "JCTools (via Netty)". This is correct: Netty shades `org.jctools` under `io/netty/util/shaded`, which is then shaded in `org/apache/iceberg/aws/shaded`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#issuecomment-4316730261

   Thanks, @kevinjqliu! I've merged this to get the checks running. There are now license updates or dependency changes to fix. See comments above.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140138564


##########
kafka-connect/kafka-connect-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,233 @@
+com.azure:azure-core-http-netty:1.16.3

Review Comment:
   Comparing this with azure-bundle, here are the deps in that bundle that are not here:
   
   ```
   com.azure:azure-security-keyvault-keys:4.10.6
   com.fasterxml.jackson.core:jackson-annotations:2.18.4
   com.fasterxml.jackson.core:jackson-core:2.18.4.1
   com.fasterxml.jackson.core:jackson-databind:2.18.4
   com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.18.4
   io.netty:netty-buffer:4.1.130.Final
   io.netty:netty-codec:4.1.130.Final
   io.netty:netty-codec-http2:4.1.130.Final
   io.netty:netty-codec-http:4.1.130.Final
   io.netty:netty-common:4.1.130.Final
   io.netty:netty-handler:4.1.130.Final
   io.netty:netty-resolver:4.1.130.Final
   io.netty:netty-transport:4.1.130.Final
   io.netty:netty-transport-classes-epoll:4.1.130.Final
   io.netty:netty-transport-native-unix-common:4.1.130.Final
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134358625


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.google.errorprone:error_prone_annotations:2.10.0
+dev.failsafe:failsafe:3.3.2
+io.airlift:aircompressor:2.0.3
+io.dropwizard.metrics:metrics-core:3.2.6
+org.apache.avro:avro:1.12.1
+org.apache.datasketches:datasketches-java:6.2.0
+org.apache.datasketches:datasketches-memory:3.0.2
+org.apache.flink:flink-metrics-dropwizard:2.1.0

Review Comment:
   This is not listed in the LICENSE file.
   
   I also wonder if this should be excluded and not added to LICENSE because it seems like something that should be included in the Flink runtime. I suspect that we need to add it as a `compileOnly` dependency or suppress it in the runtime config.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134422082


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   The LICENSE file stats that this contains many missing libraries:
   - Apache Commons [not in the Jar]
   - OpenTelemetry [not in the Jar]
   - Netty [not in the Jar]
   - Apache Arrow [not in the Jar]
   - javax.annotation-api [not in the Jar]
   - Apache Apache Thrift [shaded by Parquet]
   - Fastutil [shaded by Parquet]
   - Apache Hive [artifact from ORC?]
   - Alibaba Cloud Credentials [fixed by #15858]
   - Alibaba Cloud Tea [fixed by #15858]
   - Google GAX [fixed by #15655]
   - Google Auth Library [fixed by #15655]
   - Google Cloud BigQuery client [fixed by #15655]
   - Google Gson [fixed by #15858]
   - Google Protobuf [fixed by #15655]
   - Google API Common [fixed by #15655]
   - Google Http Client [fixed by #15655]
   - Google Auto Valve [fixed by #15655]
   - Google flatbuffers [fixed by #15655]
   - Kotlin standard library [fixed by #15858]
   - gRPC [fixed by #15858]
   - Google APIs [fixed by #15655]
   - Google Cloud APIs [fixed by #15655]
   - JSpecify [fixed by #15655]
   - Animal Sniffer Annotations [fixed by #15655]
   - Android Annotations [fixed by #15655]
   - Conscrypt [fixed by #15655]
   - Perfmark [fixed by #15655]
   - org.json. [fixed by #15655]
   - OpenSensus [fixed by #15655]
   - JaCoCo [fixed by #15858]
   - JAXB [fixed by #15858]
   - Okio [fixed by #15858]
   - OkHttp [fixed by #15858]
   - ThreeTen BP [fixed by #15655]
   
   The reason for most of the extras is that there were no LICENSE updates after a few recent PRs:
   - #15858 removed leaked transitive dependencies from Aliyun
   - #15655 removed leaked transitive dependencies from GCP
   - #15613 removed failsafe, but iceberg-aws still leaks it
   
   Also, LICENSE contains Google Guava, which is present because this shades `iceberg-bundled-guava`. But shading in that module means we don't have it listed here (FYI).
   
   Action items:
   - Find out why some libraries were there but are no longer:
       - Arrow, Netty, Apache Commons, OpenTelemetry, javax.annotations
   - Fix the Hive entry in LICENSE. Before #15449 it was clear that this was shaded by ORC. Now the only Hive reference I see is META-INF files so I think this is probably incorrect.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140349709


##########
spark/v4.0/spark-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,40 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.15.2
+com.fasterxml.jackson.core:jackson-databind:2.15.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.google.errorprone:error_prone_annotations:2.10.0
+com.google.flatbuffers:flatbuffers-java:23.5.26
+dev.failsafe:failsafe:3.3.2
+io.airlift:aircompressor:2.0.3
+io.netty:netty-buffer:4.2.12.Final
+io.netty:netty-common:4.2.12.Final
+org.apache.arrow:arrow-format:15.0.2
+org.apache.arrow:arrow-memory-core:15.0.2
+org.apache.arrow:arrow-memory-netty:15.0.2
+org.apache.arrow:arrow-vector:15.0.2
+org.apache.avro:avro:1.12.1
+org.apache.datasketches:datasketches-java:6.2.0
+org.apache.datasketches:datasketches-memory:3.0.2
+org.apache.httpcomponents.client5:httpclient5:5.6
+org.apache.httpcomponents.core5:httpcore5-h2:5.4
+org.apache.httpcomponents.core5:httpcore5:5.4
+org.apache.orc:orc-core:1.9.8
+org.apache.orc:orc-shims:1.9.8
+org.apache.parquet:parquet-avro:1.17.0
+org.apache.parquet:parquet-column:1.17.0
+org.apache.parquet:parquet-common:1.17.0
+org.apache.parquet:parquet-encoding:1.17.0
+org.apache.parquet:parquet-format-structures:1.17.0
+org.apache.parquet:parquet-hadoop:1.17.0
+org.apache.parquet:parquet-jackson:1.17.0
+org.apache.parquet:parquet-variant:1.17.0
+org.checkerframework:checker-qual:3.19.0
+org.eclipse.collections:eclipse-collections-api:11.1.0
+org.eclipse.collections:eclipse-collections:11.1.0

Review Comment:
   This is a transitive dependency from `arrow-vector`:
   
   ```
   +--- org.apache.arrow:arrow-vector:15.0.2
   |    +--- org.apache.arrow:arrow-format:15.0.2
   |    |    \--- com.google.flatbuffers:flatbuffers-java:23.5.26
   |    +--- org.apache.arrow:arrow-memory-core:15.0.2
   |    +--- com.fasterxml.jackson.core:jackson-core:2.16.0 -> 2.15.2 (*)
   |    +--- com.fasterxml.jackson.core:jackson-annotations:2.16.0 -> 2.21
   |    +--- com.fasterxml.jackson.core:jackson-databind:2.16.0 -> 2.15.2 (*)
   |    +--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.16.0 -> 2.21.2
   |    |    +--- com.fasterxml.jackson.core:jackson-annotations:2.21
   |    |    +--- com.fasterxml.jackson.core:jackson-core:2.21.2 -> 2.15.2 (*)
   |    |    +--- com.fasterxml.jackson.core:jackson-databind:2.21.2 -> 2.15.2 (*)
   |    |    \--- com.fasterxml.jackson:jackson-bom:2.21.2 (*)
   |    +--- com.google.flatbuffers:flatbuffers-java:23.5.26
   |    \--- org.eclipse.collections:eclipse-collections:11.1.0
   |         \--- org.eclipse.collections:eclipse-collections-api:11.1.0
   ```
   
   This is missing from LICENSE.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134363085


##########
flink/v2.0/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   I'm not reviewing older versions of Flink yet. I think we should determine what needs to change for the current version and then verify the same changes on the older ones.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134654205


##########
gcp-bundle/runtime-deps.txt:
##########
@@ -0,0 +1,114 @@
+com.fasterxml.jackson.core:jackson-annotations:2.18.2
+com.fasterxml.jackson.core:jackson-core:2.18.2
+com.fasterxml.jackson.core:jackson-databind:2.18.2
+com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.18.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.18.2
+com.fasterxml.woodstox:woodstox-core:7.0.0
+com.google.android:annotations:4.1.1.4
+com.google.api-client:google-api-client:2.7.2
+com.google.api.grpc:gapic-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1alpha:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta:3.24.0
+com.google.api.grpc:proto-google-cloud-kms-v1:0.182.0
+com.google.api.grpc:proto-google-cloud-monitoring-v3:3.89.0
+com.google.api.grpc:proto-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-common-protos:2.67.0
+com.google.api.grpc:proto-google-iam-v1:1.62.0
+com.google.api:api-common:2.59.0
+com.google.api:gax-grpc:2.76.0
+com.google.api:gax-httpjson:2.76.0
+com.google.api:gax:2.76.0
+com.google.apis:google-api-services-bigquery:v2-rev20251012-2.0.0
+com.google.apis:google-api-services-storage:v1-rev20260204-2.0.0
+com.google.auth:google-auth-library-credentials:1.43.0
+com.google.auth:google-auth-library-oauth2-http:1.43.0
+com.google.auto.value:auto-value-annotations:1.11.1
+com.google.cloud.gcs.analytics:client:1.2.3
+com.google.cloud.gcs.analytics:gcs-analytics-core:1.2.3
+com.google.cloud.opentelemetry:detector-resources-support:0.33.0
+com.google.cloud.opentelemetry:exporter-metrics:0.33.0
+com.google.cloud.opentelemetry:shared-resourcemapping:0.33.0
+com.google.cloud:google-cloud-bigquery:2.62.0
+com.google.cloud:google-cloud-bigquerystorage:3.24.0
+com.google.cloud:google-cloud-core-grpc:2.66.0
+com.google.cloud:google-cloud-core-http:2.66.0
+com.google.cloud:google-cloud-core:2.66.0
+com.google.cloud:google-cloud-kms:2.91.0
+com.google.cloud:google-cloud-monitoring:3.89.0
+com.google.cloud:google-cloud-storage:2.64.1
+com.google.code.findbugs:jsr305:3.0.2
+com.google.code.gson:gson:2.12.1
+com.google.errorprone:error_prone_annotations:2.42.0
+com.google.flatbuffers:flatbuffers-java:24.3.25
+com.google.guava:failureaccess:1.0.3
+com.google.guava:guava:33.5.0-jre
+com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
+com.google.http-client:google-http-client-apache-v2:2.1.0
+com.google.http-client:google-http-client-appengine:2.1.0
+com.google.http-client:google-http-client-gson:2.1.0
+com.google.http-client:google-http-client-jackson2:2.1.0
+com.google.http-client:google-http-client:2.1.0
+com.google.j2objc:j2objc-annotations:3.1
+com.google.oauth-client:google-oauth-client:1.39.0
+com.google.protobuf:protobuf-java-util:4.33.2
+com.google.protobuf:protobuf-java:4.33.2
+com.google.re2j:re2j:1.8
+commons-codec:commons-codec:1.18.0
+io.grpc:grpc-alts:1.76.3
+io.grpc:grpc-api:1.76.3
+io.grpc:grpc-auth:1.76.3
+io.grpc:grpc-context:1.76.3
+io.grpc:grpc-core:1.76.3
+io.grpc:grpc-googleapis:1.76.3
+io.grpc:grpc-grpclb:1.76.3
+io.grpc:grpc-inprocess:1.76.3
+io.grpc:grpc-netty-shaded:1.76.3
+io.grpc:grpc-opentelemetry:1.76.3
+io.grpc:grpc-protobuf-lite:1.76.3
+io.grpc:grpc-protobuf:1.76.3
+io.grpc:grpc-rls:1.76.3
+io.grpc:grpc-services:1.76.3
+io.grpc:grpc-stub:1.76.3
+io.grpc:grpc-util:1.76.3
+io.grpc:grpc-xds:1.76.3
+io.netty:netty-buffer:4.1.110.Final
+io.netty:netty-common:4.1.110.Final
+io.opencensus:opencensus-api:0.31.1
+io.opencensus:opencensus-contrib-http-util:0.31.1
+io.opentelemetry.contrib:opentelemetry-gcp-resources:1.37.0-alpha
+io.opentelemetry.semconv:opentelemetry-semconv:1.29.0-alpha
+io.opentelemetry:opentelemetry-api:1.51.0
+io.opentelemetry:opentelemetry-context:1.51.0
+io.opentelemetry:opentelemetry-sdk-common:1.51.0
+io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:1.51.0
+io.opentelemetry:opentelemetry-sdk-logs:1.51.0
+io.opentelemetry:opentelemetry-sdk-metrics:1.51.0
+io.opentelemetry:opentelemetry-sdk-trace:1.51.0
+io.opentelemetry:opentelemetry-sdk:1.51.0
+io.perfmark:perfmark-api:0.27.0
+javax.annotation:javax.annotation-api:1.3.2
+org.apache.arrow:arrow-format:17.0.0
+org.apache.arrow:arrow-memory-core:17.0.0
+org.apache.arrow:arrow-memory-netty-buffer-patch:17.0.0
+org.apache.arrow:arrow-memory-netty:17.0.0
+org.apache.arrow:arrow-vector:17.0.0
+org.apache.commons:commons-lang3:3.20.0
+org.apache.httpcomponents:httpclient:4.5.14
+org.apache.httpcomponents:httpcore:4.4.16
+org.checkerframework:checker-compat-qual:2.5.6
+org.checkerframework:checker-qual:3.49.0
+org.codehaus.mojo:animal-sniffer-annotations:1.24
+org.codehaus.woodstox:stax2-api:4.2.2
+org.conscrypt:conscrypt-openjdk-uber:2.5.2
+org.json:json:20250517
+org.jspecify:jspecify:1.0.0
+org.slf4j:slf4j-api:2.0.17
+org.threeten:threeten-extra:1.8.0
+org.threeten:threetenbp:1.7.0

Review Comment:
   The rest of these appear to be real dependencies from GCP and correctly included in the LICENSE file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#issuecomment-4309900738

   @rdblue This is the [PR to fix open-api LICENSE and NOTICE issues](https://github.com/apache/iceberg/pull/15872). Please take a look and we can discuss whether the module needs to be published there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140112659


##########
spark/v3.4/spark-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,40 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   The only difference between v4.1 runtime deps and this is the Jackson version:
   
   ```diff
   2,3c2,3
   < com.fasterxml.jackson.core:jackson-core:2.15.2
   < com.fasterxml.jackson.core:jackson-databind:2.15.2
   ---
   > com.fasterxml.jackson.core:jackson-core:2.14.2
   > com.fasterxml.jackson.core:jackson-databind:2.14.2
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140113873


##########
spark/v3.5/spark-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,40 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   Identical to the 4.1 deps.



##########
spark/v4.0/spark-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,40 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   Identical to the 4.1 deps.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140132445


##########
kafka-connect/kafka-connect-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,233 @@
+com.azure:azure-core-http-netty:1.16.3
+com.azure:azure-core:1.57.1
+com.azure:azure-identity:1.18.2
+com.azure:azure-json:1.5.1
+com.azure:azure-storage-blob:12.33.3
+com.azure:azure-storage-common:12.32.2
+com.azure:azure-storage-file-datalake:12.26.3
+com.azure:azure-storage-internal-avro:12.18.2
+com.azure:azure-xml:1.2.1
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.21.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.fasterxml.woodstox:woodstox-core:6.7.0
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.github.pjfanning:jersey-json:1.22.0
+com.google.android:annotations:4.1.1.4

Review Comment:
   Comparing this with gcp-bundle, here are the deps in that bundle but not here:
   
   ```
   com.google.api.grpc:proto-google-cloud-kms-v1
   com.google.cloud.gcs.analytics:client
   com.google.cloud.gcs.analytics:gcs-analytics-core
   com.google.cloud:google-cloud-kms
   org.jspecify:jspecify
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140430282


##########
spark/v4.0/spark-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,40 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.15.2
+com.fasterxml.jackson.core:jackson-databind:2.15.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.google.errorprone:error_prone_annotations:2.10.0
+com.google.flatbuffers:flatbuffers-java:23.5.26
+dev.failsafe:failsafe:3.3.2
+io.airlift:aircompressor:2.0.3
+io.netty:netty-buffer:4.2.12.Final
+io.netty:netty-common:4.2.12.Final
+org.apache.arrow:arrow-format:15.0.2
+org.apache.arrow:arrow-memory-core:15.0.2
+org.apache.arrow:arrow-memory-netty:15.0.2
+org.apache.arrow:arrow-vector:15.0.2
+org.apache.avro:avro:1.12.1
+org.apache.datasketches:datasketches-java:6.2.0
+org.apache.datasketches:datasketches-memory:3.0.2
+org.apache.httpcomponents.client5:httpclient5:5.6
+org.apache.httpcomponents.core5:httpcore5-h2:5.4
+org.apache.httpcomponents.core5:httpcore5:5.4
+org.apache.orc:orc-core:1.9.8
+org.apache.orc:orc-shims:1.9.8
+org.apache.parquet:parquet-avro:1.17.0
+org.apache.parquet:parquet-column:1.17.0
+org.apache.parquet:parquet-common:1.17.0
+org.apache.parquet:parquet-encoding:1.17.0
+org.apache.parquet:parquet-format-structures:1.17.0
+org.apache.parquet:parquet-hadoop:1.17.0
+org.apache.parquet:parquet-jackson:1.17.0
+org.apache.parquet:parquet-variant:1.17.0
+org.checkerframework:checker-qual:3.19.0
+org.eclipse.collections:eclipse-collections-api:11.1.0
+org.eclipse.collections:eclipse-collections:11.1.0

Review Comment:
   Apparently, this was removed in later versions according to research by my Claude agent:
   
   > Eclipse Collections was added to Arrow Java in 15.0.0 (January 2024) and removed again in 16.0.0 (April 2024) — it was only present in 15.x releases (15.0.0, 15.0.1, 15.0.2).
   
   This is not a blocker with the addition in LICENSE. But we should follow up and use a version of Arrow that doesn't include it if possible. And we should remove it from here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134422082


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   The LICENSE file stats that this contains a few missing libraries:
   - Apache Commons
   - Apache Apache Thrift
   - Apache Hive
   - Apache Arrow
   - Alibaba Cloud Credentials [fixed by #15858]
   - Alibaba Cloud Tea [fixed by #15858]
   - Google GAX
   - Google Auth Library
   - Google Cloud BigQuery client
   - Google Gson [fixed by #15858]
   - Google Guava
   - Google Protobuf
   - Google API Common
   - Google Http Client
   - Google Auto Value
   - Google flatbuffers
   - OpenTelemetry
   - Kotlin standard library [fixed by #15858]
   - gRPC
   - Netty
   - Google APIs
   - Google Cloud APIs
   - javax.annotation-api
   - JSpecify
   - Animal Sniffer Annotations
   - Android Annotations
   - Conscrypt
   - Perfmark
   - org.json.
   - OpenSensus
   - JaCoCo [fixed by #15858]
   - JAXB [fixed by #15858]
   - Okio [fixed by #15858]
   - OkHttp [fixed by #15858]
   - ThreeTen BP
   - Fastutil
   
   There was no LICENSE update after a few recent PRs:
   - #15858 removed leaked transitive dependencies from Aliyun
   - #15655 removed leaked transitive dependencies from GCP
   - #15613 removed failsafe, but iceberg-aws still leaks it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140132445


##########
kafka-connect/kafka-connect-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,233 @@
+com.azure:azure-core-http-netty:1.16.3
+com.azure:azure-core:1.57.1
+com.azure:azure-identity:1.18.2
+com.azure:azure-json:1.5.1
+com.azure:azure-storage-blob:12.33.3
+com.azure:azure-storage-common:12.32.2
+com.azure:azure-storage-file-datalake:12.26.3
+com.azure:azure-storage-internal-avro:12.18.2
+com.azure:azure-xml:1.2.1
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.21.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.fasterxml.woodstox:woodstox-core:6.7.0
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.github.pjfanning:jersey-json:1.22.0
+com.google.android:annotations:4.1.1.4

Review Comment:
   Comparing this with gcp-bundle here are the deps in that bundle but not here:
   
   ```
   com.fasterxml.jackson.core:jackson-annotations:2.18.2
   com.fasterxml.jackson.core:jackson-core:2.18.2
   com.fasterxml.jackson.core:jackson-databind:2.18.2
   com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.18.2
   com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.18.2
   com.fasterxml.woodstox:woodstox-core:7.0.0
   com.google.api.grpc:proto-google-cloud-kms-v1:0.182.0
   com.google.auto.value:auto-value-annotations:1.11.1
   com.google.cloud.gcs.analytics:client:1.2.3
   com.google.cloud.gcs.analytics:gcs-analytics-core:1.2.3
   com.google.cloud:google-cloud-kms:2.91.0
   commons-codec:commons-codec:1.18.0
   io.grpc:grpc-netty-shaded:1.76.3
   io.netty:netty-buffer:4.1.110.Final
   io.netty:netty-common:4.1.110.Final
   org.apache.commons:commons-lang3:3.20.0
   org.jspecify:jspecify:1.0.0
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134419187


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.google.errorprone:error_prone_annotations:2.10.0
+dev.failsafe:failsafe:3.3.2

Review Comment:
   This is leaked by iceberg-aws and should not be bundled.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134422082


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   The LICENSE file stats that this contains many missing libraries:
   - Apache Commons [not in the Jar]
   - OpenTelemetry [not in the Jar]
   - Netty [not in the Jar]
   - Apache Arrow [not in the Jar]
   - javax.annotation-api [not in the Jar]
   - Apache Apache Thrift [shaded by Parquet]
   - Fastutil [shaded by Parquet]
   - Apache Hive [artifact from ORC?]
   - Alibaba Cloud Credentials [fixed by #15858]
   - Alibaba Cloud Tea [fixed by #15858]
   - Google GAX [fixed by #15655]
   - Google Auth Library [fixed by #15655]
   - Google Cloud BigQuery client [fixed by #15655]
   - Google Gson [fixed by #15858]
   - Google Protobuf [fixed by #15655]
   - Google API Common [fixed by #15655]
   - Google Http Client [fixed by #15655]
   - Google Auto Valve [fixed by #15655]
   - Google flatbuffers [fixed by #15655]
   - Kotlin standard library [fixed by #15858]
   - gRPC [fixed by #15858]
   - Google APIs [fixed by #15655]
   - Google Cloud APIs [fixed by #15655]
   - JSpecify [fixed by #15655]
   - Animal Sniffer Annotations [fixed by #15655]
   - Android Annotations [fixed by #15655]
   - Conscrypt [fixed by #15655]
   - Perfmark [fixed by #15655]
   - org.json. [fixed by #15655]
   - OpenSensus [fixed by #15655]
   - JaCoCo [fixed by #15858]
   - JAXB [fixed by #15858]
   - Okio [fixed by #15858]
   - OkHttp [fixed by #15858]
   - ThreeTen BP [fixed by #15655]
   
   The reason for most of the extras is that there were no LICENSE updates after a few recent PRs:
   - #15858 removed leaked transitive dependencies from Aliyun
   - #15655 removed leaked transitive dependencies from GCP
   - #15613 removed failsafe, but iceberg-aws still leaks it
   
   Also, LICENSE contains Google Guava, which is present because this shades `iceberg-bundled-guava`. But shading in that module means we don't have it listed here (FYI).
   
   Action items:
   - Find out why some libraries were there but are no longer:
       - Arrow, Netty, Apache Commons, OpenTelemetry, javax.annotations
   - Fix the Hive entry in LICENSE. Before #15449 it was clear that this was shaded by ORC. Now the only Hive reference I see is META-INF files so I think this is probably incorrect.
   - Remove all of the fixed dependency leaks from LICENSE and NOTICE



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] ci: add all `runtime-deps.txt` files [iceberg]

Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#issuecomment-4301151125

   no `open-api` module?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3139576648


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   OpenTelemetry, Arrow, Netty, and javax.annotations appear to have never been included in the runtime Jar through dependencies. It's unclear why they were added to LICENSE by fb2c8ac but they are not part of the runtime now and can be removed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134547582


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.google.errorprone:error_prone_annotations:2.10.0
+dev.failsafe:failsafe:3.3.2

Review Comment:
   This is used directly by `S3InputStream`, which means it needs to be included when `iceberg-aws` is included becuase it is not provided by the AWS dependencies. I don't think this is a good reason to keep using it and that we should replace it with `Tasks`, unless it is doing something special.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] ci: add all `runtime-deps.txt` files [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#issuecomment-4306228463

   Should we be publishing the iceberg-open-api module? It is for testing so it would make sense to me if we didn't publish one.
   
   I also didn't know about that runtime module and we will need to evaluate its LICENSE and NOTICE files before including it in any more releases.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134525009


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.google.errorprone:error_prone_annotations:2.10.0
+dev.failsafe:failsafe:3.3.2
+io.airlift:aircompressor:2.0.3
+io.dropwizard.metrics:metrics-core:3.2.6
+org.apache.avro:avro:1.12.1
+org.apache.datasketches:datasketches-java:6.2.0
+org.apache.datasketches:datasketches-memory:3.0.2
+org.apache.flink:flink-metrics-dropwizard:2.1.0

Review Comment:
   Opened a PR for this fix: https://github.com/apache/iceberg/pull/16093



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3139708790


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   Opened #16102 to fix the LICENSE and NOTICE for v2.1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140546439


##########
aws-bundle/runtime-deps.txt:
##########
@@ -0,0 +1,70 @@
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.google.errorprone:error_prone_annotations:2.10.0
+commons-codec:commons-codec:1.17.1
+commons-logging:commons-logging:1.2
+io.netty:netty-buffer:4.1.132.Final
+io.netty:netty-codec-http2:4.1.132.Final
+io.netty:netty-codec-http:4.1.132.Final
+io.netty:netty-codec:4.1.132.Final
+io.netty:netty-common:4.1.132.Final
+io.netty:netty-handler:4.1.132.Final
+io.netty:netty-resolver:4.1.132.Final
+io.netty:netty-transport-classes-epoll:4.1.132.Final
+io.netty:netty-transport-native-unix-common:4.1.132.Final
+io.netty:netty-transport:4.1.132.Final
+org.apache.httpcomponents:httpclient:4.5.13
+org.apache.httpcomponents:httpcore:4.4.16
+org.apache.logging.log4j:log4j-api:2.20.0
+org.apache.logging.log4j:log4j-core:2.20.0
+org.apache.logging.log4j:log4j-slf4j-impl:2.20.0

Review Comment:
   Opened #16105 to fix this by removing log4j.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140127792


##########
kafka-connect/kafka-connect-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,233 @@
+com.azure:azure-core-http-netty:1.16.3
+com.azure:azure-core:1.57.1
+com.azure:azure-identity:1.18.2
+com.azure:azure-json:1.5.1
+com.azure:azure-storage-blob:12.33.3
+com.azure:azure-storage-common:12.32.2
+com.azure:azure-storage-file-datalake:12.26.3
+com.azure:azure-storage-internal-avro:12.18.2
+com.azure:azure-xml:1.2.1
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.21.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.fasterxml.woodstox:woodstox-core:6.7.0
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.github.pjfanning:jersey-json:1.22.0
+com.google.android:annotations:4.1.1.4
+com.google.api-client:google-api-client:2.7.2
+com.google.api.grpc:gapic-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1alpha:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta:3.24.0
+com.google.api.grpc:proto-google-cloud-monitoring-v3:3.89.0
+com.google.api.grpc:proto-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-common-protos:2.67.0
+com.google.api.grpc:proto-google-iam-v1:1.62.0
+com.google.api:api-common:2.59.0
+com.google.api:gax-grpc:2.76.0
+com.google.api:gax-httpjson:2.76.0
+com.google.api:gax:2.76.0
+com.google.apis:google-api-services-bigquery:v2-rev20251012-2.0.0
+com.google.apis:google-api-services-storage:v1-rev20260204-2.0.0
+com.google.auth:google-auth-library-credentials:1.43.0
+com.google.auth:google-auth-library-oauth2-http:1.43.0
+com.google.auto.value:auto-value-annotations:1.11.0
+com.google.cloud.opentelemetry:detector-resources-support:0.33.0
+com.google.cloud.opentelemetry:exporter-metrics:0.33.0
+com.google.cloud.opentelemetry:shared-resourcemapping:0.33.0
+com.google.cloud:google-cloud-bigquery:2.62.0
+com.google.cloud:google-cloud-bigquerystorage:3.24.0
+com.google.cloud:google-cloud-core-grpc:2.66.0
+com.google.cloud:google-cloud-core-http:2.66.0
+com.google.cloud:google-cloud-core:2.66.0
+com.google.cloud:google-cloud-monitoring:3.89.0
+com.google.cloud:google-cloud-storage:2.64.1
+com.google.code.findbugs:jsr305:3.0.2
+com.google.code.gson:gson:2.12.1
+com.google.errorprone:error_prone_annotations:2.42.0
+com.google.flatbuffers:flatbuffers-java:24.3.25
+com.google.guava:failureaccess:1.0.3
+com.google.guava:guava:33.5.0-jre
+com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
+com.google.http-client:google-http-client-apache-v2:2.1.0
+com.google.http-client:google-http-client-appengine:2.1.0
+com.google.http-client:google-http-client-gson:2.1.0
+com.google.http-client:google-http-client-jackson2:2.1.0
+com.google.http-client:google-http-client:2.1.0
+com.google.j2objc:j2objc-annotations:3.1
+com.google.oauth-client:google-oauth-client:1.39.0
+com.google.protobuf:protobuf-java-util:4.33.2
+com.google.protobuf:protobuf-java:4.33.2
+com.google.re2j:re2j:1.8
+com.jcraft:jsch:0.1.55
+com.microsoft.azure:msal4j-persistence-extension:1.3.0
+com.microsoft.azure:msal4j:1.23.1
+com.sun.xml.bind:jaxb-impl:2.2.3-1
+commons-cli:commons-cli:1.9.0
+commons-codec:commons-codec:1.19.0
+commons-io:commons-io:2.20.0
+commons-logging:commons-logging:1.2
+commons-net:commons-net:3.9.0
+commons-pool:commons-pool:1.6
+dev.failsafe:failsafe:3.3.2
+dnsjava:dnsjava:3.6.1
+io.airlift:aircompressor:2.0.3
+io.dropwizard.metrics:metrics-core:3.2.4
+io.grpc:grpc-alts:1.76.3
+io.grpc:grpc-api:1.76.3
+io.grpc:grpc-auth:1.76.3
+io.grpc:grpc-context:1.76.3
+io.grpc:grpc-core:1.76.3
+io.grpc:grpc-googleapis:1.76.3
+io.grpc:grpc-grpclb:1.76.3
+io.grpc:grpc-inprocess:1.76.3
+io.grpc:grpc-netty-shaded:1.80.0
+io.grpc:grpc-opentelemetry:1.76.3
+io.grpc:grpc-protobuf-lite:1.76.3
+io.grpc:grpc-protobuf:1.76.3
+io.grpc:grpc-rls:1.76.3
+io.grpc:grpc-services:1.76.3
+io.grpc:grpc-stub:1.76.3
+io.grpc:grpc-util:1.76.3
+io.grpc:grpc-xds:1.76.3
+io.netty:netty-buffer:4.1.132.Final
+io.netty:netty-codec-dns:4.1.128.Final
+io.netty:netty-codec-http2:4.1.132.Final
+io.netty:netty-codec-http:4.1.132.Final
+io.netty:netty-codec-socks:4.1.130.Final
+io.netty:netty-codec:4.1.132.Final
+io.netty:netty-common:4.1.132.Final
+io.netty:netty-handler-proxy:4.1.130.Final
+io.netty:netty-handler:4.1.132.Final
+io.netty:netty-resolver-dns-classes-macos:4.1.128.Final
+io.netty:netty-resolver-dns-native-macos:4.1.128.Final
+io.netty:netty-resolver-dns:4.1.128.Final
+io.netty:netty-resolver:4.1.132.Final
+io.netty:netty-tcnative-boringssl-static:2.0.74.Final
+io.netty:netty-tcnative-classes:2.0.74.Final
+io.netty:netty-transport-classes-epoll:4.1.132.Final
+io.netty:netty-transport-classes-kqueue:4.1.130.Final
+io.netty:netty-transport-native-epoll:4.1.130.Final
+io.netty:netty-transport-native-kqueue:4.1.130.Final
+io.netty:netty-transport-native-unix-common:4.1.132.Final
+io.netty:netty-transport:4.1.132.Final
+io.opencensus:opencensus-api:0.31.1
+io.opencensus:opencensus-contrib-http-util:0.31.1
+io.opentelemetry.contrib:opentelemetry-gcp-resources:1.37.0-alpha
+io.opentelemetry.semconv:opentelemetry-semconv:1.29.0-alpha
+io.opentelemetry:opentelemetry-api:1.51.0
+io.opentelemetry:opentelemetry-context:1.51.0
+io.opentelemetry:opentelemetry-sdk-common:1.51.0
+io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:1.51.0
+io.opentelemetry:opentelemetry-sdk-logs:1.51.0
+io.opentelemetry:opentelemetry-sdk-metrics:1.51.0
+io.opentelemetry:opentelemetry-sdk-trace:1.51.0
+io.opentelemetry:opentelemetry-sdk:1.51.0
+io.perfmark:perfmark-api:0.27.0
+io.projectreactor.netty:reactor-netty-core:1.2.13
+io.projectreactor.netty:reactor-netty-http:1.2.13
+io.projectreactor:reactor-core:3.7.14
+jakarta.activation:jakarta.activation-api:1.2.1
+javax.annotation:javax.annotation-api:1.3.2
+javax.servlet.jsp:jsp-api:2.1
+javax.servlet:javax.servlet-api:3.1.0
+javax.xml.bind:jaxb-api:2.2.2
+javax.xml.stream:stax-api:1.0-2
+net.java.dev.jna:jna-platform:5.17.0
+net.java.dev.jna:jna:5.17.0
+org.apache.arrow:arrow-format:17.0.0
+org.apache.arrow:arrow-memory-core:17.0.0
+org.apache.arrow:arrow-memory-netty-buffer-patch:17.0.0
+org.apache.arrow:arrow-memory-netty:17.0.0
+org.apache.arrow:arrow-vector:17.0.0
+org.apache.avro:avro:1.12.1
+org.apache.commons:commons-collections4:4.4
+org.apache.commons:commons-compress:1.28.0
+org.apache.commons:commons-lang3:3.18.0
+org.apache.commons:commons-math3:3.6.1
+org.apache.commons:commons-text:1.14.0
+org.apache.hadoop.thirdparty:hadoop-shaded-guava:1.5.0
+org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_25:1.5.0
+org.apache.hadoop:hadoop-annotations:3.4.3
+org.apache.hadoop:hadoop-common:3.4.3
+org.apache.httpcomponents.client5:httpclient5:5.6
+org.apache.httpcomponents.core5:httpcore5-h2:5.4
+org.apache.httpcomponents.core5:httpcore5:5.4
+org.apache.httpcomponents:httpclient:4.5.14
+org.apache.httpcomponents:httpcore:4.4.16
+org.apache.orc:orc-core:1.9.8
+org.apache.orc:orc-shims:1.9.8
+org.apache.parquet:parquet-avro:1.17.0
+org.apache.parquet:parquet-column:1.17.0
+org.apache.parquet:parquet-common:1.17.0
+org.apache.parquet:parquet-encoding:1.17.0
+org.apache.parquet:parquet-format-structures:1.17.0
+org.apache.parquet:parquet-hadoop:1.17.0
+org.apache.parquet:parquet-jackson:1.17.0
+org.apache.parquet:parquet-variant:1.17.0
+org.bouncycastle:bcprov-jdk18on:1.82
+org.checkerframework:checker-compat-qual:2.5.6
+org.checkerframework:checker-qual:3.49.0
+org.codehaus.jettison:jettison:1.5.5
+org.codehaus.mojo:animal-sniffer-annotations:1.24
+org.codehaus.woodstox:stax2-api:4.2.2
+org.conscrypt:conscrypt-openjdk-uber:2.5.2
+org.json:json:20250517
+org.locationtech.jts:jts-core:1.20.0
+org.mongodb:bson:4.11.5
+org.reactivestreams:reactive-streams:1.0.4
+org.roaringbitmap:RoaringBitmap:1.6.14
+org.slf4j:slf4j-api:2.0.17
+org.threeten:threeten-extra:1.8.0
+org.threeten:threetenbp:1.7.0
+org.xerial.snappy:snappy-java:1.1.10.8
+software.amazon.awssdk.crt:aws-crt:0.43.9

Review Comment:
   Comparing this with aws-bundle's runtime deps, here are the deps in aws but not this bundle:
   
   ```
   org.apache.logging.log4j:log4j-api
   org.apache.logging.log4j:log4j-core
   org.apache.logging.log4j:log4j-slf4j-impl
   software.amazon.awssdk:cloudwatch
   software.amazon.awssdk:cloudwatch-metric-publisher
   software.amazon.awssdk:s3control
   software.amazon.awssdk:smithy-rpcv2-protocol
   software.amazon.awssdk:third-party-jackson-dataformat-cbor
   software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin
   software.amazon.s3.analyticsaccelerator:analyticsaccelerator-s3
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140138564


##########
kafka-connect/kafka-connect-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,233 @@
+com.azure:azure-core-http-netty:1.16.3

Review Comment:
   Comparing this with azure-bundle, here are the deps in that bundle that are not here:
   
   ```
   com.azure:azure-security-keyvault-keys
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134642489


##########
gcp-bundle/runtime-deps.txt:
##########
@@ -0,0 +1,114 @@
+com.fasterxml.jackson.core:jackson-annotations:2.18.2
+com.fasterxml.jackson.core:jackson-core:2.18.2
+com.fasterxml.jackson.core:jackson-databind:2.18.2
+com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.18.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.18.2
+com.fasterxml.woodstox:woodstox-core:7.0.0
+com.google.android:annotations:4.1.1.4
+com.google.api-client:google-api-client:2.7.2
+com.google.api.grpc:gapic-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1alpha:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta:3.24.0
+com.google.api.grpc:proto-google-cloud-kms-v1:0.182.0
+com.google.api.grpc:proto-google-cloud-monitoring-v3:3.89.0
+com.google.api.grpc:proto-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-common-protos:2.67.0
+com.google.api.grpc:proto-google-iam-v1:1.62.0
+com.google.api:api-common:2.59.0
+com.google.api:gax-grpc:2.76.0
+com.google.api:gax-httpjson:2.76.0
+com.google.api:gax:2.76.0
+com.google.apis:google-api-services-bigquery:v2-rev20251012-2.0.0
+com.google.apis:google-api-services-storage:v1-rev20260204-2.0.0
+com.google.auth:google-auth-library-credentials:1.43.0
+com.google.auth:google-auth-library-oauth2-http:1.43.0
+com.google.auto.value:auto-value-annotations:1.11.1
+com.google.cloud.gcs.analytics:client:1.2.3
+com.google.cloud.gcs.analytics:gcs-analytics-core:1.2.3
+com.google.cloud.opentelemetry:detector-resources-support:0.33.0
+com.google.cloud.opentelemetry:exporter-metrics:0.33.0
+com.google.cloud.opentelemetry:shared-resourcemapping:0.33.0
+com.google.cloud:google-cloud-bigquery:2.62.0
+com.google.cloud:google-cloud-bigquerystorage:3.24.0
+com.google.cloud:google-cloud-core-grpc:2.66.0
+com.google.cloud:google-cloud-core-http:2.66.0
+com.google.cloud:google-cloud-core:2.66.0
+com.google.cloud:google-cloud-kms:2.91.0
+com.google.cloud:google-cloud-monitoring:3.89.0
+com.google.cloud:google-cloud-storage:2.64.1
+com.google.code.findbugs:jsr305:3.0.2

Review Comment:
   Findbugs is excluded throughout the codebase because it was originally LGPL and cannot be bundled. The license issues weren't clarified, and a clean implementation was created: https://github.com/stephenc/findbugs-annotations
   
   Although the maven metadata reports ALv2, we need to exclude it. If we need the annotations (which are not required to function), then we should use the stephenc verison.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3139897336


##########
flink/v2.0/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   This file and the 2.1 file are the same. I also verified that all of the directories/packages in the Jars are the same. We should be able to copy the 2.1 LICENSE and NOTICE into 2.0.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140351185


##########
spark/v4.0/spark-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,40 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.15.2
+com.fasterxml.jackson.core:jackson-databind:2.15.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.google.errorprone:error_prone_annotations:2.10.0
+com.google.flatbuffers:flatbuffers-java:23.5.26
+dev.failsafe:failsafe:3.3.2
+io.airlift:aircompressor:2.0.3
+io.netty:netty-buffer:4.2.12.Final
+io.netty:netty-common:4.2.12.Final
+org.apache.arrow:arrow-format:15.0.2
+org.apache.arrow:arrow-memory-core:15.0.2
+org.apache.arrow:arrow-memory-netty:15.0.2
+org.apache.arrow:arrow-vector:15.0.2
+org.apache.avro:avro:1.12.1
+org.apache.datasketches:datasketches-java:6.2.0
+org.apache.datasketches:datasketches-memory:3.0.2
+org.apache.httpcomponents.client5:httpclient5:5.6
+org.apache.httpcomponents.core5:httpcore5-h2:5.4
+org.apache.httpcomponents.core5:httpcore5:5.4
+org.apache.orc:orc-core:1.9.8
+org.apache.orc:orc-shims:1.9.8
+org.apache.parquet:parquet-avro:1.17.0
+org.apache.parquet:parquet-column:1.17.0
+org.apache.parquet:parquet-common:1.17.0
+org.apache.parquet:parquet-encoding:1.17.0
+org.apache.parquet:parquet-format-structures:1.17.0
+org.apache.parquet:parquet-hadoop:1.17.0
+org.apache.parquet:parquet-jackson:1.17.0
+org.apache.parquet:parquet-variant:1.17.0
+org.checkerframework:checker-qual:3.19.0
+org.eclipse.collections:eclipse-collections-api:11.1.0
+org.eclipse.collections:eclipse-collections:11.1.0
+org.eclipse.microprofile.openapi:microprofile-openapi-api:4.1.1

Review Comment:
   Transitive dependency from Nessie



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134089878


##########
aws-bundle/runtime-deps.txt:
##########
@@ -0,0 +1,70 @@
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.google.errorprone:error_prone_annotations:2.10.0
+commons-codec:commons-codec:1.17.1
+commons-logging:commons-logging:1.2
+io.netty:netty-buffer:4.1.132.Final
+io.netty:netty-codec-http2:4.1.132.Final
+io.netty:netty-codec-http:4.1.132.Final
+io.netty:netty-codec:4.1.132.Final
+io.netty:netty-common:4.1.132.Final
+io.netty:netty-handler:4.1.132.Final
+io.netty:netty-resolver:4.1.132.Final
+io.netty:netty-transport-classes-epoll:4.1.132.Final
+io.netty:netty-transport-native-unix-common:4.1.132.Final
+io.netty:netty-transport:4.1.132.Final
+org.apache.httpcomponents:httpclient:4.5.13
+org.apache.httpcomponents:httpcore:4.4.16
+org.apache.logging.log4j:log4j-api:2.20.0
+org.apache.logging.log4j:log4j-core:2.20.0
+org.apache.logging.log4j:log4j-slf4j-impl:2.20.0

Review Comment:
   Log4J is not included in the LICENSE file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140416947


##########
spark/v4.0/spark-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,40 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.15.2
+com.fasterxml.jackson.core:jackson-databind:2.15.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.google.errorprone:error_prone_annotations:2.10.0
+com.google.flatbuffers:flatbuffers-java:23.5.26
+dev.failsafe:failsafe:3.3.2
+io.airlift:aircompressor:2.0.3
+io.netty:netty-buffer:4.2.12.Final
+io.netty:netty-common:4.2.12.Final
+org.apache.arrow:arrow-format:15.0.2
+org.apache.arrow:arrow-memory-core:15.0.2
+org.apache.arrow:arrow-memory-netty:15.0.2
+org.apache.arrow:arrow-vector:15.0.2
+org.apache.avro:avro:1.12.1
+org.apache.datasketches:datasketches-java:6.2.0
+org.apache.datasketches:datasketches-memory:3.0.2
+org.apache.httpcomponents.client5:httpclient5:5.6
+org.apache.httpcomponents.core5:httpcore5-h2:5.4
+org.apache.httpcomponents.core5:httpcore5:5.4
+org.apache.orc:orc-core:1.9.8
+org.apache.orc:orc-shims:1.9.8
+org.apache.parquet:parquet-avro:1.17.0
+org.apache.parquet:parquet-column:1.17.0
+org.apache.parquet:parquet-common:1.17.0
+org.apache.parquet:parquet-encoding:1.17.0
+org.apache.parquet:parquet-format-structures:1.17.0
+org.apache.parquet:parquet-hadoop:1.17.0
+org.apache.parquet:parquet-jackson:1.17.0
+org.apache.parquet:parquet-variant:1.17.0
+org.checkerframework:checker-qual:3.19.0
+org.eclipse.collections:eclipse-collections-api:11.1.0
+org.eclipse.collections:eclipse-collections:11.1.0

Review Comment:
   Opened #16104 with the LICENSE and NOTICE updates.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3139708790


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   Opened #16120 to fix the LICENSE and NOTICE for v2.1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3139904874


##########
flink/v1.20/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   As with 2.0, this file and the directories/packages in the Jar are the same as v2.1, so we should be able to use the same LICENSE and NOTICE.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140127792


##########
kafka-connect/kafka-connect-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,233 @@
+com.azure:azure-core-http-netty:1.16.3
+com.azure:azure-core:1.57.1
+com.azure:azure-identity:1.18.2
+com.azure:azure-json:1.5.1
+com.azure:azure-storage-blob:12.33.3
+com.azure:azure-storage-common:12.32.2
+com.azure:azure-storage-file-datalake:12.26.3
+com.azure:azure-storage-internal-avro:12.18.2
+com.azure:azure-xml:1.2.1
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.21.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.fasterxml.woodstox:woodstox-core:6.7.0
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.github.pjfanning:jersey-json:1.22.0
+com.google.android:annotations:4.1.1.4
+com.google.api-client:google-api-client:2.7.2
+com.google.api.grpc:gapic-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1alpha:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta:3.24.0
+com.google.api.grpc:proto-google-cloud-monitoring-v3:3.89.0
+com.google.api.grpc:proto-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-common-protos:2.67.0
+com.google.api.grpc:proto-google-iam-v1:1.62.0
+com.google.api:api-common:2.59.0
+com.google.api:gax-grpc:2.76.0
+com.google.api:gax-httpjson:2.76.0
+com.google.api:gax:2.76.0
+com.google.apis:google-api-services-bigquery:v2-rev20251012-2.0.0
+com.google.apis:google-api-services-storage:v1-rev20260204-2.0.0
+com.google.auth:google-auth-library-credentials:1.43.0
+com.google.auth:google-auth-library-oauth2-http:1.43.0
+com.google.auto.value:auto-value-annotations:1.11.0
+com.google.cloud.opentelemetry:detector-resources-support:0.33.0
+com.google.cloud.opentelemetry:exporter-metrics:0.33.0
+com.google.cloud.opentelemetry:shared-resourcemapping:0.33.0
+com.google.cloud:google-cloud-bigquery:2.62.0
+com.google.cloud:google-cloud-bigquerystorage:3.24.0
+com.google.cloud:google-cloud-core-grpc:2.66.0
+com.google.cloud:google-cloud-core-http:2.66.0
+com.google.cloud:google-cloud-core:2.66.0
+com.google.cloud:google-cloud-monitoring:3.89.0
+com.google.cloud:google-cloud-storage:2.64.1
+com.google.code.findbugs:jsr305:3.0.2
+com.google.code.gson:gson:2.12.1
+com.google.errorprone:error_prone_annotations:2.42.0
+com.google.flatbuffers:flatbuffers-java:24.3.25
+com.google.guava:failureaccess:1.0.3
+com.google.guava:guava:33.5.0-jre
+com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
+com.google.http-client:google-http-client-apache-v2:2.1.0
+com.google.http-client:google-http-client-appengine:2.1.0
+com.google.http-client:google-http-client-gson:2.1.0
+com.google.http-client:google-http-client-jackson2:2.1.0
+com.google.http-client:google-http-client:2.1.0
+com.google.j2objc:j2objc-annotations:3.1
+com.google.oauth-client:google-oauth-client:1.39.0
+com.google.protobuf:protobuf-java-util:4.33.2
+com.google.protobuf:protobuf-java:4.33.2
+com.google.re2j:re2j:1.8
+com.jcraft:jsch:0.1.55
+com.microsoft.azure:msal4j-persistence-extension:1.3.0
+com.microsoft.azure:msal4j:1.23.1
+com.sun.xml.bind:jaxb-impl:2.2.3-1
+commons-cli:commons-cli:1.9.0
+commons-codec:commons-codec:1.19.0
+commons-io:commons-io:2.20.0
+commons-logging:commons-logging:1.2
+commons-net:commons-net:3.9.0
+commons-pool:commons-pool:1.6
+dev.failsafe:failsafe:3.3.2
+dnsjava:dnsjava:3.6.1
+io.airlift:aircompressor:2.0.3
+io.dropwizard.metrics:metrics-core:3.2.4
+io.grpc:grpc-alts:1.76.3
+io.grpc:grpc-api:1.76.3
+io.grpc:grpc-auth:1.76.3
+io.grpc:grpc-context:1.76.3
+io.grpc:grpc-core:1.76.3
+io.grpc:grpc-googleapis:1.76.3
+io.grpc:grpc-grpclb:1.76.3
+io.grpc:grpc-inprocess:1.76.3
+io.grpc:grpc-netty-shaded:1.80.0
+io.grpc:grpc-opentelemetry:1.76.3
+io.grpc:grpc-protobuf-lite:1.76.3
+io.grpc:grpc-protobuf:1.76.3
+io.grpc:grpc-rls:1.76.3
+io.grpc:grpc-services:1.76.3
+io.grpc:grpc-stub:1.76.3
+io.grpc:grpc-util:1.76.3
+io.grpc:grpc-xds:1.76.3
+io.netty:netty-buffer:4.1.132.Final
+io.netty:netty-codec-dns:4.1.128.Final
+io.netty:netty-codec-http2:4.1.132.Final
+io.netty:netty-codec-http:4.1.132.Final
+io.netty:netty-codec-socks:4.1.130.Final
+io.netty:netty-codec:4.1.132.Final
+io.netty:netty-common:4.1.132.Final
+io.netty:netty-handler-proxy:4.1.130.Final
+io.netty:netty-handler:4.1.132.Final
+io.netty:netty-resolver-dns-classes-macos:4.1.128.Final
+io.netty:netty-resolver-dns-native-macos:4.1.128.Final
+io.netty:netty-resolver-dns:4.1.128.Final
+io.netty:netty-resolver:4.1.132.Final
+io.netty:netty-tcnative-boringssl-static:2.0.74.Final
+io.netty:netty-tcnative-classes:2.0.74.Final
+io.netty:netty-transport-classes-epoll:4.1.132.Final
+io.netty:netty-transport-classes-kqueue:4.1.130.Final
+io.netty:netty-transport-native-epoll:4.1.130.Final
+io.netty:netty-transport-native-kqueue:4.1.130.Final
+io.netty:netty-transport-native-unix-common:4.1.132.Final
+io.netty:netty-transport:4.1.132.Final
+io.opencensus:opencensus-api:0.31.1
+io.opencensus:opencensus-contrib-http-util:0.31.1
+io.opentelemetry.contrib:opentelemetry-gcp-resources:1.37.0-alpha
+io.opentelemetry.semconv:opentelemetry-semconv:1.29.0-alpha
+io.opentelemetry:opentelemetry-api:1.51.0
+io.opentelemetry:opentelemetry-context:1.51.0
+io.opentelemetry:opentelemetry-sdk-common:1.51.0
+io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:1.51.0
+io.opentelemetry:opentelemetry-sdk-logs:1.51.0
+io.opentelemetry:opentelemetry-sdk-metrics:1.51.0
+io.opentelemetry:opentelemetry-sdk-trace:1.51.0
+io.opentelemetry:opentelemetry-sdk:1.51.0
+io.perfmark:perfmark-api:0.27.0
+io.projectreactor.netty:reactor-netty-core:1.2.13
+io.projectreactor.netty:reactor-netty-http:1.2.13
+io.projectreactor:reactor-core:3.7.14
+jakarta.activation:jakarta.activation-api:1.2.1
+javax.annotation:javax.annotation-api:1.3.2
+javax.servlet.jsp:jsp-api:2.1
+javax.servlet:javax.servlet-api:3.1.0
+javax.xml.bind:jaxb-api:2.2.2
+javax.xml.stream:stax-api:1.0-2
+net.java.dev.jna:jna-platform:5.17.0
+net.java.dev.jna:jna:5.17.0
+org.apache.arrow:arrow-format:17.0.0
+org.apache.arrow:arrow-memory-core:17.0.0
+org.apache.arrow:arrow-memory-netty-buffer-patch:17.0.0
+org.apache.arrow:arrow-memory-netty:17.0.0
+org.apache.arrow:arrow-vector:17.0.0
+org.apache.avro:avro:1.12.1
+org.apache.commons:commons-collections4:4.4
+org.apache.commons:commons-compress:1.28.0
+org.apache.commons:commons-lang3:3.18.0
+org.apache.commons:commons-math3:3.6.1
+org.apache.commons:commons-text:1.14.0
+org.apache.hadoop.thirdparty:hadoop-shaded-guava:1.5.0
+org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_25:1.5.0
+org.apache.hadoop:hadoop-annotations:3.4.3
+org.apache.hadoop:hadoop-common:3.4.3
+org.apache.httpcomponents.client5:httpclient5:5.6
+org.apache.httpcomponents.core5:httpcore5-h2:5.4
+org.apache.httpcomponents.core5:httpcore5:5.4
+org.apache.httpcomponents:httpclient:4.5.14
+org.apache.httpcomponents:httpcore:4.4.16
+org.apache.orc:orc-core:1.9.8
+org.apache.orc:orc-shims:1.9.8
+org.apache.parquet:parquet-avro:1.17.0
+org.apache.parquet:parquet-column:1.17.0
+org.apache.parquet:parquet-common:1.17.0
+org.apache.parquet:parquet-encoding:1.17.0
+org.apache.parquet:parquet-format-structures:1.17.0
+org.apache.parquet:parquet-hadoop:1.17.0
+org.apache.parquet:parquet-jackson:1.17.0
+org.apache.parquet:parquet-variant:1.17.0
+org.bouncycastle:bcprov-jdk18on:1.82
+org.checkerframework:checker-compat-qual:2.5.6
+org.checkerframework:checker-qual:3.49.0
+org.codehaus.jettison:jettison:1.5.5
+org.codehaus.mojo:animal-sniffer-annotations:1.24
+org.codehaus.woodstox:stax2-api:4.2.2
+org.conscrypt:conscrypt-openjdk-uber:2.5.2
+org.json:json:20250517
+org.locationtech.jts:jts-core:1.20.0
+org.mongodb:bson:4.11.5
+org.reactivestreams:reactive-streams:1.0.4
+org.roaringbitmap:RoaringBitmap:1.6.14
+org.slf4j:slf4j-api:2.0.17
+org.threeten:threeten-extra:1.8.0
+org.threeten:threetenbp:1.7.0
+org.xerial.snappy:snappy-java:1.1.10.8
+software.amazon.awssdk.crt:aws-crt:0.43.9

Review Comment:
   Comparing this with aws-bundle's runtime deps, here are the deps in aws but not this bundle:
   
   ```
   com.google.errorprone:error_prone_annotations:2.10.0
   commons-codec:commons-codec:1.17.1
   org.apache.httpcomponents:httpclient:4.5.13
   org.apache.logging.log4j:log4j-api:2.20.0
   org.apache.logging.log4j:log4j-core:2.20.0
   org.apache.logging.log4j:log4j-slf4j-impl:2.20.0
   org.checkerframework:checker-qual:3.19.0
   software.amazon.awssdk:cloudwatch:2.42.33
   software.amazon.awssdk:cloudwatch-metric-publisher:2.42.33
   software.amazon.awssdk:s3control:2.42.33
   software.amazon.awssdk:smithy-rpcv2-protocol:2.42.33
   software.amazon.awssdk:third-party-jackson-dataformat-cbor:2.42.33
   software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin:2.4.1
   software.amazon.s3.analyticsaccelerator:analyticsaccelerator-s3:1.3.1
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3140170831


##########
kafka-connect/kafka-connect-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,233 @@
+com.azure:azure-core-http-netty:1.16.3
+com.azure:azure-core:1.57.1
+com.azure:azure-identity:1.18.2
+com.azure:azure-json:1.5.1
+com.azure:azure-storage-blob:12.33.3
+com.azure:azure-storage-common:12.32.2
+com.azure:azure-storage-file-datalake:12.26.3
+com.azure:azure-storage-internal-avro:12.18.2
+com.azure:azure-xml:1.2.1
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.21.2
+com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2
+com.fasterxml.woodstox:woodstox-core:6.7.0
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.github.pjfanning:jersey-json:1.22.0
+com.google.android:annotations:4.1.1.4
+com.google.api-client:google-api-client:2.7.2
+com.google.api.grpc:gapic-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1alpha:3.24.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.196.0
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta:3.24.0
+com.google.api.grpc:proto-google-cloud-monitoring-v3:3.89.0
+com.google.api.grpc:proto-google-cloud-storage-v2:2.64.1
+com.google.api.grpc:proto-google-common-protos:2.67.0
+com.google.api.grpc:proto-google-iam-v1:1.62.0
+com.google.api:api-common:2.59.0
+com.google.api:gax-grpc:2.76.0
+com.google.api:gax-httpjson:2.76.0
+com.google.api:gax:2.76.0
+com.google.apis:google-api-services-bigquery:v2-rev20251012-2.0.0
+com.google.apis:google-api-services-storage:v1-rev20260204-2.0.0
+com.google.auth:google-auth-library-credentials:1.43.0
+com.google.auth:google-auth-library-oauth2-http:1.43.0
+com.google.auto.value:auto-value-annotations:1.11.0
+com.google.cloud.opentelemetry:detector-resources-support:0.33.0
+com.google.cloud.opentelemetry:exporter-metrics:0.33.0
+com.google.cloud.opentelemetry:shared-resourcemapping:0.33.0
+com.google.cloud:google-cloud-bigquery:2.62.0
+com.google.cloud:google-cloud-bigquerystorage:3.24.0
+com.google.cloud:google-cloud-core-grpc:2.66.0
+com.google.cloud:google-cloud-core-http:2.66.0
+com.google.cloud:google-cloud-core:2.66.0
+com.google.cloud:google-cloud-monitoring:3.89.0
+com.google.cloud:google-cloud-storage:2.64.1
+com.google.code.findbugs:jsr305:3.0.2
+com.google.code.gson:gson:2.12.1
+com.google.errorprone:error_prone_annotations:2.42.0
+com.google.flatbuffers:flatbuffers-java:24.3.25
+com.google.guava:failureaccess:1.0.3
+com.google.guava:guava:33.5.0-jre
+com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
+com.google.http-client:google-http-client-apache-v2:2.1.0
+com.google.http-client:google-http-client-appengine:2.1.0
+com.google.http-client:google-http-client-gson:2.1.0
+com.google.http-client:google-http-client-jackson2:2.1.0
+com.google.http-client:google-http-client:2.1.0
+com.google.j2objc:j2objc-annotations:3.1
+com.google.oauth-client:google-oauth-client:1.39.0
+com.google.protobuf:protobuf-java-util:4.33.2
+com.google.protobuf:protobuf-java:4.33.2
+com.google.re2j:re2j:1.8
+com.jcraft:jsch:0.1.55
+com.microsoft.azure:msal4j-persistence-extension:1.3.0
+com.microsoft.azure:msal4j:1.23.1
+com.sun.xml.bind:jaxb-impl:2.2.3-1
+commons-cli:commons-cli:1.9.0
+commons-codec:commons-codec:1.19.0
+commons-io:commons-io:2.20.0
+commons-logging:commons-logging:1.2
+commons-net:commons-net:3.9.0
+commons-pool:commons-pool:1.6
+dev.failsafe:failsafe:3.3.2
+dnsjava:dnsjava:3.6.1
+io.airlift:aircompressor:2.0.3
+io.dropwizard.metrics:metrics-core:3.2.4
+io.grpc:grpc-alts:1.76.3
+io.grpc:grpc-api:1.76.3
+io.grpc:grpc-auth:1.76.3
+io.grpc:grpc-context:1.76.3
+io.grpc:grpc-core:1.76.3
+io.grpc:grpc-googleapis:1.76.3
+io.grpc:grpc-grpclb:1.76.3
+io.grpc:grpc-inprocess:1.76.3
+io.grpc:grpc-netty-shaded:1.80.0
+io.grpc:grpc-opentelemetry:1.76.3
+io.grpc:grpc-protobuf-lite:1.76.3
+io.grpc:grpc-protobuf:1.76.3
+io.grpc:grpc-rls:1.76.3
+io.grpc:grpc-services:1.76.3
+io.grpc:grpc-stub:1.76.3
+io.grpc:grpc-util:1.76.3
+io.grpc:grpc-xds:1.76.3
+io.netty:netty-buffer:4.1.132.Final
+io.netty:netty-codec-dns:4.1.128.Final
+io.netty:netty-codec-http2:4.1.132.Final
+io.netty:netty-codec-http:4.1.132.Final
+io.netty:netty-codec-socks:4.1.130.Final
+io.netty:netty-codec:4.1.132.Final
+io.netty:netty-common:4.1.132.Final
+io.netty:netty-handler-proxy:4.1.130.Final
+io.netty:netty-handler:4.1.132.Final
+io.netty:netty-resolver-dns-classes-macos:4.1.128.Final
+io.netty:netty-resolver-dns-native-macos:4.1.128.Final
+io.netty:netty-resolver-dns:4.1.128.Final
+io.netty:netty-resolver:4.1.132.Final
+io.netty:netty-tcnative-boringssl-static:2.0.74.Final
+io.netty:netty-tcnative-classes:2.0.74.Final
+io.netty:netty-transport-classes-epoll:4.1.132.Final
+io.netty:netty-transport-classes-kqueue:4.1.130.Final
+io.netty:netty-transport-native-epoll:4.1.130.Final
+io.netty:netty-transport-native-kqueue:4.1.130.Final
+io.netty:netty-transport-native-unix-common:4.1.132.Final
+io.netty:netty-transport:4.1.132.Final
+io.opencensus:opencensus-api:0.31.1
+io.opencensus:opencensus-contrib-http-util:0.31.1
+io.opentelemetry.contrib:opentelemetry-gcp-resources:1.37.0-alpha
+io.opentelemetry.semconv:opentelemetry-semconv:1.29.0-alpha
+io.opentelemetry:opentelemetry-api:1.51.0
+io.opentelemetry:opentelemetry-context:1.51.0
+io.opentelemetry:opentelemetry-sdk-common:1.51.0
+io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:1.51.0
+io.opentelemetry:opentelemetry-sdk-logs:1.51.0
+io.opentelemetry:opentelemetry-sdk-metrics:1.51.0
+io.opentelemetry:opentelemetry-sdk-trace:1.51.0
+io.opentelemetry:opentelemetry-sdk:1.51.0
+io.perfmark:perfmark-api:0.27.0
+io.projectreactor.netty:reactor-netty-core:1.2.13
+io.projectreactor.netty:reactor-netty-http:1.2.13
+io.projectreactor:reactor-core:3.7.14
+jakarta.activation:jakarta.activation-api:1.2.1
+javax.annotation:javax.annotation-api:1.3.2
+javax.servlet.jsp:jsp-api:2.1
+javax.servlet:javax.servlet-api:3.1.0
+javax.xml.bind:jaxb-api:2.2.2
+javax.xml.stream:stax-api:1.0-2
+net.java.dev.jna:jna-platform:5.17.0
+net.java.dev.jna:jna:5.17.0
+org.apache.arrow:arrow-format:17.0.0

Review Comment:
   Here are the dependencies in this runtime that aren't in any of the cloud provider bundles:
   
   ```
   com.github.luben:zstd-jni
   com.github.pjfanning:jersey-json
   com.jcraft:jsch
   commons-cli:commons-cli
   commons-io:commons-io
   commons-net:commons-net
   commons-pool:commons-pool
   com.sun.xml.bind:jaxb-impl
   dev.failsafe:failsafe
   dnsjava:dnsjava
   io.airlift:aircompressor
   io.dropwizard.metrics:metrics-core
   jakarta.activation:jakarta.activation-api
   javax.servlet:javax.servlet-api
   javax.servlet.jsp:jsp-api
   javax.xml.bind:jaxb-api
   javax.xml.stream:stax-api
   org.apache.avro:avro
   org.apache.commons:commons-collections4
   org.apache.commons:commons-compress
   org.apache.commons:commons-math3
   org.apache.commons:commons-text
   org.apache.hadoop:hadoop-annotations
   org.apache.hadoop:hadoop-common
   org.apache.hadoop.thirdparty:hadoop-shaded-guava
   org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_25
   org.apache.httpcomponents.client5:httpclient5
   org.apache.httpcomponents.core5:httpcore5
   org.apache.httpcomponents.core5:httpcore5-h2
   org.apache.orc:orc-core
   org.apache.orc:orc-shims
   org.apache.parquet:parquet-avro
   org.apache.parquet:parquet-column
   org.apache.parquet:parquet-common
   org.apache.parquet:parquet-encoding
   org.apache.parquet:parquet-format-structures
   org.apache.parquet:parquet-hadoop
   org.apache.parquet:parquet-jackson
   org.apache.parquet:parquet-variant
   org.bouncycastle:bcprov-jdk18on
   org.codehaus.jettison:jettison
   org.locationtech.jts:jts-core
   org.mongodb:bson
   org.roaringbitmap:RoaringBitmap
   org.xerial.snappy:snappy-java
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue merged PR #16081:
URL: https://github.com/apache/iceberg/pull/16081


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134598230


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21

Review Comment:
   Apache Commons is from this commit: https://github.com/apache/iceberg/commit/760a20bc26b332b490cef72e56921c8056d3c5ca
   
   #2102 copied array methods into `ArrayUtil`. This isn't a big problem, but it doesn't seem worth the hassle of tracking it down in LICENSE to have array copy methods. The implementations don't match project style or provide value. A good first issue is to remove them.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134096841


##########
aws-bundle/runtime-deps.txt:
##########
@@ -0,0 +1,70 @@
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.google.errorprone:error_prone_annotations:2.10.0
+commons-codec:commons-codec:1.17.1
+commons-logging:commons-logging:1.2
+io.netty:netty-buffer:4.1.132.Final
+io.netty:netty-codec-http2:4.1.132.Final
+io.netty:netty-codec-http:4.1.132.Final
+io.netty:netty-codec:4.1.132.Final
+io.netty:netty-common:4.1.132.Final
+io.netty:netty-handler:4.1.132.Final
+io.netty:netty-resolver:4.1.132.Final
+io.netty:netty-transport-classes-epoll:4.1.132.Final
+io.netty:netty-transport-native-unix-common:4.1.132.Final
+io.netty:netty-transport:4.1.132.Final
+org.apache.httpcomponents:httpclient:4.5.13
+org.apache.httpcomponents:httpcore:4.4.16
+org.apache.logging.log4j:log4j-api:2.20.0
+org.apache.logging.log4j:log4j-core:2.20.0
+org.apache.logging.log4j:log4j-slf4j-impl:2.20.0
+org.checkerframework:checker-qual:3.19.0
+org.reactivestreams:reactive-streams:1.0.4
+org.slf4j:slf4j-api:2.0.17
+software.amazon.awssdk.crt:aws-crt:0.43.9
+software.amazon.awssdk:annotations:2.42.33
+software.amazon.awssdk:apache-client:2.42.33
+software.amazon.awssdk:arns:2.42.33
+software.amazon.awssdk:auth:2.42.33
+software.amazon.awssdk:aws-core:2.42.33
+software.amazon.awssdk:aws-json-protocol:2.42.33
+software.amazon.awssdk:aws-query-protocol:2.42.33
+software.amazon.awssdk:aws-xml-protocol:2.42.33
+software.amazon.awssdk:checksums-spi:2.42.33
+software.amazon.awssdk:checksums:2.42.33
+software.amazon.awssdk:cloudwatch-metric-publisher:2.42.33
+software.amazon.awssdk:cloudwatch:2.42.33
+software.amazon.awssdk:crt-core:2.42.33
+software.amazon.awssdk:dynamodb:2.42.33
+software.amazon.awssdk:endpoints-spi:2.42.33
+software.amazon.awssdk:glue:2.42.33
+software.amazon.awssdk:http-auth-aws-crt:2.42.33
+software.amazon.awssdk:http-auth-aws-eventstream:2.42.33
+software.amazon.awssdk:http-auth-aws:2.42.33
+software.amazon.awssdk:http-auth-spi:2.42.33
+software.amazon.awssdk:http-auth:2.42.33
+software.amazon.awssdk:http-client-spi:2.42.33
+software.amazon.awssdk:iam:2.42.33
+software.amazon.awssdk:identity-spi:2.42.33
+software.amazon.awssdk:json-utils:2.42.33
+software.amazon.awssdk:kms:2.42.33
+software.amazon.awssdk:lakeformation:2.42.33
+software.amazon.awssdk:metrics-spi:2.42.33
+software.amazon.awssdk:netty-nio-client:2.42.33
+software.amazon.awssdk:profiles:2.42.33
+software.amazon.awssdk:protocol-core:2.42.33
+software.amazon.awssdk:regions:2.42.33
+software.amazon.awssdk:retries-spi:2.42.33
+software.amazon.awssdk:retries:2.42.33
+software.amazon.awssdk:s3:2.42.33
+software.amazon.awssdk:s3control:2.42.33
+software.amazon.awssdk:sdk-core:2.42.33
+software.amazon.awssdk:smithy-rpcv2-protocol:2.42.33
+software.amazon.awssdk:sso:2.42.33
+software.amazon.awssdk:sts:2.42.33
+software.amazon.awssdk:third-party-jackson-core:2.42.33
+software.amazon.awssdk:third-party-jackson-dataformat-cbor:2.42.33
+software.amazon.awssdk:utils-lite:2.42.33
+software.amazon.awssdk:utils:2.42.33
+software.amazon.eventstream:eventstream:1.0.1
+software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin:2.4.1
+software.amazon.s3.analyticsaccelerator:analyticsaccelerator-s3:1.3.1

Review Comment:
   I think that these are fine. I didn't check every one against the latest update (https://github.com/apache/iceberg/commit/fb2c8ac3faf) because they are now grouped into high level SDK modules, but they are all ALv2 and should be okay.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] CI: Add runtime-deps.txt files for all runtimes and bundles [iceberg]

Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16081:
URL: https://github.com/apache/iceberg/pull/16081#discussion_r3134547582


##########
flink/v2.1/flink-runtime/runtime-deps.txt:
##########
@@ -0,0 +1,33 @@
+com.fasterxml.jackson.core:jackson-annotations:2.21
+com.fasterxml.jackson.core:jackson-core:2.21.2
+com.fasterxml.jackson.core:jackson-databind:2.21.2
+com.github.ben-manes.caffeine:caffeine:2.9.3
+com.github.luben:zstd-jni:1.5.7-3
+com.google.errorprone:error_prone_annotations:2.10.0
+dev.failsafe:failsafe:3.3.2

Review Comment:
   This is used directly by `S3InputStream`, which means it needs to be included when `iceberg-aws` is included becuase it is not provided by the AWS dependencies. I don't think this is a good reason to keep using it and that we should replace it with `Tasks`, unless it is doing something special.
   
   Since this is in the license docs, I think this isn't a blocker for 1.11.0 or 1.10.2, but we should remove it to keep dependencies to a minimum.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org