You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 13:44:45 UTC
[GitHub] [beam] damccorm opened a new issue, #19846: Vendor or shade the dependencies of beam-sdks-java-io-google-cloud-platform
damccorm opened a new issue, #19846:
URL: https://github.com/apache/beam/issues/19846
Here at Spotify, we regularly face conflicting dependencies issues, usually caused by guava, grpc-* and netty-*. Looking at this bug tracker, it seems that we are not alone and that conflicting versions are a recurring issue.
In `beam-sdks-java-core,` `beam-runners-direct-java` and `beam-runners-google-cloud-dataflow-java` those dependencies are vendored to avoid such issue.
I suggest to do the same for `beam-sdks-java-io-google-cloud-platform`.
FIY, here's the list of all the dependencies pulled by `beam-sdks-java-io-google-cloud-platform`:
`com.fasterxml.jackson.core:jackson-annotations:2.9.10:default`
`com.fasterxml.jackson.core:jackson-core:2.9.10:default`
`com.fasterxml.jackson.core:jackson-databind:2.9.10:default`
`com.google.api:api-common:1.7.0:default`
`com.google.api:gax:1.38.0:default`
`com.google.api:gax-grpc:1.38.0:default`
`com.google.api:gax-httpjson:0.52.0:default`
`com.google.api-client:google-api-client:1.27.0:default`
`com.google.api-client:google-api-client-jackson2:1.27.0:default`
`com.google.api-client:google-api-client-java6:1.27.0:default`
`com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.44.0:default`
`com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:0.38.0:default`
`com.google.api.grpc:grpc-google-cloud-bigtable-v2:0.38.0:default`
`com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.43.0:default`
`com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:1.6.0:default`
`com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:1.6.0:default`
`com.google.api.grpc:grpc-google-cloud-spanner-v1:1.6.0:default`
`com.google.api.grpc:grpc-google-common-protos:1.12.0:default`
`com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.44.0:default`
`com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:0.38.0:default`
`com.google.api.grpc:proto-google-cloud-bigtable-v2:0.44.0:default`
`com.google.api.grpc:proto-google-cloud-datastore-v1:0.44.0:default`
`com.google.api.grpc:proto-google-cloud-pubsub-v1:1.43.0:default`
`com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.6.0:default`
`com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.6.0:default`
`com.google.api.grpc:proto-google-cloud-spanner-v1:1.6.0:default`
`com.google.api.grpc:proto-google-common-protos:1.12.0:default`
`com.google.api.grpc:proto-google-iam-v1:0.12.0:default`
`com.google.apis:google-api-services-bigquery:v2-rev20181104-1.27.0:default`
`com.google.apis:google-api-services-cloudresourcemanager:v1-rev20181015-1.27.0:default`
`com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0:default`
`com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0:default`
`com.google.auth:google-auth-library-credentials:0.12.0:default`
`com.google.auth:google-auth-library-oauth2-http:0.12.0:default`
`com.google.auto.value:auto-value-annotations:1.6.3:default`
`com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha:default`
`com.google.cloud:google-cloud-bigtable:0.73.0-alpha:default`
`com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha:default`
`com.google.cloud:google-cloud-core:1.61.0:default`
`com.google.cloud:google-cloud-core-grpc:1.61.0:default`
`com.google.cloud:google-cloud-core-http:1.55.0:default`
`com.google.cloud:google-cloud-spanner:1.6.0:default`
`com.google.cloud.bigdataoss:gcsio:1.9.16:default`
`com.google.cloud.bigdataoss:util:1.9.16:default`
`com.google.cloud.bigtable:bigtable-client-core:1.8.0:default`
`com.google.cloud.datastore:datastore-v1-proto-client:1.6.0:default`
`com.google.code.findbugs:jsr305:3.0.2:default`
`com.google.code.gson:gson:2.7:default`
`com.google.errorprone:error_prone_annotations:2.3.2:default`
`com.google.flogger:flogger:0.3.1:default`
`com.google.flogger:flogger-system-backend:0.3.1:default`
`com.google.flogger:google-extensions:0.3.1:default`
`com.google.guava:failureaccess:1.0.1:default`
`com.google.guava:guava:27.0.1-jre:default`
`com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava:default`
`com.google.http-client:google-http-client:1.27.0:default`
`com.google.http-client:google-http-client-appengine:1.27.0:default`
`com.google.http-client:google-http-client-jackson:1.20.0:default`
`com.google.http-client:google-http-client-jackson2:1.27.0:default`
`com.google.http-client:google-http-client-protobuf:1.20.0:default`
`com.google.j2objc:j2objc-annotations:1.1:default`
`com.google.oauth-client:google-oauth-client:1.27.0:default`
`com.google.oauth-client:google-oauth-client-java6:1.27.0:default`
`com.google.protobuf:protobuf-java:3.6.1:default`
`com.google.protobuf:protobuf-java-util:3.6.1:default`
`com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-5:default`
`com.squareup.okhttp:okhttp:2.5.0:default`
`com.squareup.okio:okio:1.13.0:default`
`com.thoughtworks.paranamer:paranamer:2.7:default`
`commons-codec:commons-codec:1.10:default`
`commons-logging:commons-logging:1.2:default`
`io.dropwizard.metrics:metrics-core:3.1.2:default`
`io.grpc:grpc-all:1.17.1:default`
`io.grpc:grpc-alts:1.17.1:default`
`io.grpc:grpc-auth:1.17.1:default`
`io.grpc:grpc-context:1.17.1:default`
`io.grpc:grpc-core:1.17.1:default`
`io.grpc:grpc-grpclb:1.17.1:default`
`io.grpc:grpc-netty:1.17.1:default`
`io.grpc:grpc-netty-shaded:1.17.1:default`
`io.grpc:grpc-okhttp:1.17.1:default`
`io.grpc:grpc-protobuf:1.17.1:default`
`io.grpc:grpc-protobuf-lite:1.17.1:default`
`io.grpc:grpc-protobuf-nano:1.17.1:default`
`io.grpc:grpc-stub:1.17.1:default`
`io.grpc:grpc-testing:1.17.1:default`
`io.netty:netty-buffer:4.1.30.Final:default`
`io.netty:netty-codec:4.1.30.Final:default`
`io.netty:netty-codec-http:4.1.30.Final:default`
`io.netty:netty-codec-http2:4.1.30.Final:default`
`io.netty:netty-codec-socks:4.1.30.Final:default`
`io.netty:netty-common:4.1.30.Final:default`
`io.netty:netty-handler:4.1.30.Final:default`
`io.netty:netty-handler-proxy:4.1.30.Final:default`
`io.netty:netty-resolver:4.1.30.Final:default`
`io.netty:netty-tcnative-boringssl-static:2.0.17.Final:default`
`io.netty:netty-transport:4.1.30.Final:default`
`io.opencensus:opencensus-api:0.17.0:default`
`io.opencensus:opencensus-contrib-grpc-metrics:0.17.0:default`
`io.opencensus:opencensus-contrib-grpc-util:0.17.0:default`
`io.opencensus:opencensus-contrib-http-util:0.15.0:default`
`javax.annotation:javax.annotation-api:1.2:default`
`joda-time:joda-time:2.10.3:default`
`junit:junit:4.12:default`
`org.apache.avro:avro:1.8.2:default`
`org.apache.beam:beam-model-job-management:2.16.0:default`
`org.apache.beam:beam-model-pipeline:2.16.0:default`
`org.apache.beam:beam-sdks-java-core:2.16.0:default`
`org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.16.0:default`
`org.apache.beam:beam-sdks-java-extensions-protobuf:2.16.0:default`
`org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0:default`
`org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1:default`
`org.apache.beam:beam-vendor-grpc-1_21_0:0.1:default`
`org.apache.beam:beam-vendor-guava-26_0-jre:0.1:default`
`org.apache.commons:commons-compress:1.8.1:default`
`org.apache.commons:commons-lang3:3.5:default`
`org.apache.httpcomponents:httpclient:4.5.5:default`
`org.apache.httpcomponents:httpcore:4.4.9:default`
`org.checkerframework:checker-qual:2.5.2:default`
`org.codehaus.jackson:jackson-core-asl:1.9.13:default`
`org.codehaus.jackson:jackson-mapper-asl:1.9.13:default`
`org.codehaus.mojo:animal-sniffer-annotations:1.17:default`
`org.hamcrest:hamcrest-core:1.3:default`
`org.mockito:mockito-core:1.9.5:default`
`org.objenesis:objenesis:1.0:default`
`org.slf4j:slf4j-api:1.7.25:default`
`org.threeten:threetenbp:1.3.3:default`
`org.tukaani:xz:1.8:default`
`org.xerial.snappy:snappy-java:1.1.4:default`
Imported from Jira [BEAM-8497](https://issues.apache.org/jira/browse/BEAM-8497). Original Jira may contain additional context.
Reported by: jto.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #19846: Vendor or shade the dependencies of beam-sdks-java-io-google-cloud-platform
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #19846:
URL: https://github.com/apache/beam/issues/19846#issuecomment-1319407080
@lukecwik @apilloud relevant to recent issues with these deps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lukecwik commented on issue #19846: Vendor or shade the dependencies of beam-sdks-java-io-google-cloud-platform
Posted by GitBox <gi...@apache.org>.
lukecwik commented on issue #19846:
URL: https://github.com/apache/beam/issues/19846#issuecomment-1320466571
The original Beam issue was created before Apache Beam migrated to using BOMs to aid with dependency management. For example users should use the appropriate GCP libraries BOM that matches the Apache Beam version they want to use. For example:
Apache Beam 2.43.0 uses com.google.cloud:libraries-bom:26.1.3
https://github.com/apache/beam/blob/release-2.43.0/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L608
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lukecwik closed issue #19846: Vendor or shade the dependencies of beam-sdks-java-io-google-cloud-platform
Posted by GitBox <gi...@apache.org>.
lukecwik closed issue #19846: Vendor or shade the dependencies of beam-sdks-java-io-google-cloud-platform
URL: https://github.com/apache/beam/issues/19846
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org