You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by GitBox <gi...@apache.org> on 2020/03/18 13:37:38 UTC
[GitHub] [beam] suztomo opened a new pull request #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
suztomo opened a new pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156
- To use BOM, declaring the BOM dependency with "platform" ([Gradle: Importing Maven BOMs](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import))
- Removing version part from Google Cloud dependencies that is in the BOM.
- [List of Google Cloud libraries in the GCP Libraries BOM](https://storage.googleapis.com/cloud-opensource-java-dashboard/com.google.cloud/libraries-bom/4.2.0/artifact_details.html)
- I found `io.grpc:grpc-protobuf` and `io.grpc:grpc-protobuf-lite` are unused.
- The versionless coordinates cannot be used for `force` function in Gradle.
# Linkage Check Result
No change. https://gist.github.com/suztomo/3402967bd465516f01a18a80984fb4c0
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771821
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394408735
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
@lukecwik Do you still need this `force` of the dependency versions? (I found your commit in 2017 https://github.com/apache/beam/commit/aee22a9dcf9284cd362f6ed91a43e1c68cd65be3 )
I assume this `force` was introduced to pass the compilation and tests by resolving the exact versions of the dependencies when building Beam. However Beam users (both Gradle and Maven) do not have this version-pinning effect when using Beam. Therefore this `force` might have hidden dependency problems that Beam developers could discover when running tests.
Do you still need this `force`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539614
Run Java HadoopFormatIO Performance Test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394466904
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
As of now we don't have plan to increase the BOM coverage to keep maintenance cost low. Low maintenance burden leads frequent release cycle. When we find the burden is still low, we may add them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo edited a comment on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo edited a comment on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606154369
`:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest` failed: https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/252/
```
11:12:20 > Task :runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
11:13:55
11:13:55 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerWithMultipleTimerFamilyUnbounded FAILED
11:13:55 java.lang.RuntimeException at ParDoTest.java:4770
11:13:55 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4770
11:29:11
11:29:11 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerFamilyEventTimeUnbounded FAILED
11:29:11 java.lang.RuntimeException at ParDoTest.java:4708
11:29:11 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4708
```
It seems other post commit jobs are also failing:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/4738/console
```
08:01:43 > Task :runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
08:03:14
08:03:14 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerWithMultipleTimerFamilyUnbounded FAILED
08:03:14 java.lang.RuntimeException at ParDoTest.java:4770
08:03:14 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4770
08:16:38
08:16:38 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerFamilyEventTimeUnbounded FAILED
08:16:38 java.lang.RuntimeException at ParDoTest.java:4708
08:16:38 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4708
08:33:31
08:33:31 org.apache.beam.sdk.transforms.FlattenTest > testFlattenPCollectionsThenParDo FAILED
08:33:31 java.lang.RuntimeException at FlattenTest.java:118
```
Checking which PR started this failure. -> Found https://github.com/apache/beam/pull/11216#issuecomment-606158291
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539817
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] iemejia commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
iemejia commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600666394
Run Java HadoopFormatIO Performance Test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394420078
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
This artifact is not in GCP Libraries BOM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394535755
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Yes, I'll try that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539526
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606154369
`:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest` failed: https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/252/
```
11:12:20 > Task :runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
11:13:55
11:13:55 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerWithMultipleTimerFamilyUnbounded FAILED
11:13:55 java.lang.RuntimeException at ParDoTest.java:4770
11:13:55 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4770
11:29:11
11:29:11 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerFamilyEventTimeUnbounded FAILED
11:29:11 java.lang.RuntimeException at ParDoTest.java:4708
11:29:11 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4708
```
It seems other post commit jobs are also failing:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/4738/console
```
08:01:43 > Task :runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
08:03:14
08:03:14 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerWithMultipleTimerFamilyUnbounded FAILED
08:03:14 java.lang.RuntimeException at ParDoTest.java:4770
08:03:14 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4770
08:16:38
08:16:38 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests > testTimerFamilyEventTimeUnbounded FAILED
08:16:38 java.lang.RuntimeException at ParDoTest.java:4708
08:16:38 Caused by: java.lang.IllegalArgumentException at ParDoTest.java:4708
08:33:31
08:33:31 org.apache.beam.sdk.transforms.FlattenTest > testFlattenPCollectionsThenParDo FAILED
08:33:31 java.lang.RuntimeException at FlattenTest.java:118
```
Checking which PR started this failure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394427660
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
Review comment:
Thats a nice dashboard.
Any reason why the apiary generated libs are not included in the GCP BOM?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394427922
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
Can missing GCP libs be added?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-602040903
Yes, I will summarize the version change on Monday.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603540005
Run SQL Postcommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] iemejia commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
iemejia commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600666184
bang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394484244
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
> > to increase the compatibility across multiple Beam modules
>
> That increased compatibility only affects Beam's build time. Not for Beam users who consume Beam's Maven artifacts. (Correct me if I'm wrong)
Yes, that is my understanding as well.
> > How would the BOM interact with the force if it stayed or if it was removed?
>
> The question whether force is not directly relevant to the BOM (although artifacts that rely on the BOM do not have versions in this file, therefore cannot be used with `force`)
Would it be wise to enumerate all the artifacts in the BOM and `force` their resolution?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aromanenko-dev commented on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752336
Run SQL Postcommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600868999
## Java Precommit failed
`org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapperTest` is BEAM-9470.
Samza error: https://builds.apache.org/job/beam_PreCommit_Java_Commit/10414/testReport/junit/org.apache.beam.runners.samza.runtime/SamzaTimerInternalsFactoryTest/testRestore/
```
Caused by: org.rocksdb.RocksDBException: Can't access /000370.sst: IO error: while stat a file for size: /tmp/store2/000370.sst: No such file or directory
```
## Python PreCommit failed
https://builds.apache.org/job/beam_PreCommit_Python_Commit/11782/testReport/junit/apache_beam.runners.portability.fn_api_runner_test/FnApiRunnerTestWithGrpc/test_large_elements/ complains `IndexError: index out of range`.
I find another irrelevant build ([11780](https://builds.apache.org/job/beam_PreCommit_Python_Commit/11780/testReport/junit/apache_beam.runners.portability.fn_api_runner_test/FnApiRunnerTestWithGrpc/test_large_elements/)) also show this error.
They all seem irrelevant to this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394460130
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
> to increase the compatibility across multiple Beam modules
That increased compatibility only affects Beam's build time. Not for Beam users who consume Beam's Maven artifacts. (Correct me if I'm wrong)
> How would the BOM interact with the force if it stayed or if it was removed?
The question whether force is not directly relevant to the BOM (although artifacts that rely on the BOM do not have versions in this file, therefore cannot be used with `force`)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] chamikaramj commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402658146
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
Have we confirmed that GCS artifacts not included in this BOM do not conflict with artifacts included in the BOM ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r395911140
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
Review comment:
A question for my learning. Previously this file had specific dependency versions. For any released Beam version we could check the file in the release branch and get a list of dependencies and their versions.
After this change, how can we do the same thing? Would it happen through a generated BOM file in the release branch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394527172
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
> Would it be wise to enumerate all the artifacts in the BOM and force their resolution?
I couldn't find a way to do that. Even if there's a way, I feel forcing the all library versions for builds is unnecessary.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aromanenko-dev commented on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600751588
Run Java HadoopFormatIO Performance Test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771990
Run Spark ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aromanenko-dev commented on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752230
Run Spark ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771894
Run BigQueryIO Streaming Performance Test Java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] iemejia commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
iemejia commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600666561
Run Spark ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600770827
@aromanenko-dev Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394420883
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
- google_cloud_core : "com.google.cloud:google-cloud-core:$google_cloud_core_version",
- google_cloud_core_grpc : "com.google.cloud:google-cloud-core-grpc:$google_cloud_core_version",
- google_cloud_datacatalog_v1beta1 : "com.google.cloud:google-cloud-datacatalog:$google_cloud_datacatalog_version",
+ google_cloud_core : "com.google.cloud:google-cloud-core",
+ google_cloud_core_grpc : "com.google.cloud:google-cloud-core-grpc",
+ google_cloud_datacatalog_v1beta1 : "com.google.cloud:google-cloud-datacatalog",
google_cloud_dataflow_java_proto_library_all: "com.google.cloud.dataflow:google-cloud-dataflow-java-proto-library-all:0.5.160304",
google_cloud_datastore_v1_proto_client : "com.google.cloud.datastore:datastore-v1-proto-client:1.6.3",
Review comment:
This artifact is not in GCP Libraries BOM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539923
Run Spark ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604772021
Run SQL Postcommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600933706
@aaltay Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603540099
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] chamikaramj commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402660255
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Please force findings here just for the record.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r396593515
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
Review comment:
OK, this is good.
- For easy access could we link to that dashboard from the place we are including the bom in BeamModulePlugin.groovy
One clarification question. Beam will depend on that exact list of dependencies shown in that dashboard. Is that correct?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057311
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600933550
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600669458
@iemejia Thank you. Sonatype is unhealthy now. I'll check later today.
```
10:51:05 > Could not get resource 'https://oss.sonatype.org/content/repositories/staging/io/grpc/grpc-core/maven-metadata.xml'.
10:51:05 > Could not GET 'https://oss.sonatype.org/content/repositories/staging/io/grpc/grpc-core/maven-metadata.xml'.
10:51:05 > Read timed out
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] chamikaramj commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402666322
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
GCP libraries you already commented about that are not included in the BOM. For example " google_cloud_bigtable_client_core". I guess it should be fine if linkage checker passes. Note that our existing integration tests are not that useful here since they mostly check one GCS service at a time while users may use a combination.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] iemejia commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
iemejia commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600666613
Run SQL Postcommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-602716275
> @aaltay Added "Artifact Version Changes" section in the PR description.
Thanks. @chamikaramj - could you review this section, especially the IOs in downgrade/upgrade sections?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r396647354
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
Review comment:
Added the link and comments ` // google_cloud_platform_libraries_bom sets version`. They may look too verbose, but they'll help future question "_where is the version of this artifact?_", especially when we start using other BOMs than GCP Libraries BOM in future.
> Beam will depend on that exact list of dependencies shown in that dashboard.
Not exact list. Beam will pick up the version from the BOM for Maven artifacts without a version.
For example,
- Beam will depend on the BOM's gax version "1.54.0", because Beam's dependency `com.google.api:gax` does not specify a version.
- Beam **will not** depend on the BOM's Guava version "28.2-android", because Beam explicitly specify Guava's version "25.1-jre" as in `com.google.guava:guava:$guava_version`.
- Beam **will not** depend on the BOM's `com.google.http-client:google-http-client-apache-v2:1.34.2`, because Beam does not depend on `com.google.http-client:google-http-client-apache-v2`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539728
Run BigQueryIO Streaming Performance Test Java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402716462
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
Yes, Linkage Checker detects the discrepancy if there's any.
> Note that our existing integration tests are not that useful here since they mostly check one GCP service at a time while users may use a combination.
This change is not intended to fix compatibility with users' libraries. It's about how Beam maintains the versions of these Google libraries with ease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394426074
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Gradle's default resolution rules are to use "latest" when there is a conflict. We have been using `force` to ensure that we honor the versions in this top level map over any others that are suggested via transitive dependencies to increase the compatibility across multiple Beam modules.
How would the BOM interact with the `force` if it stayed or if it was removed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394535077
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Try to remove it everywhere and see what happens for the tests but that should be decoupled from this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aromanenko-dev commented on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752160
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402662384
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
Which library are you referring to?
There’s no additional conflicts. Linkage Checker verified that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] chamikaramj commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402666322
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
GCP libraries you already commented about that are not included in the BOM. For example " google_cloud_bigtable_client_core". I guess it should be fine if linkage checker passes. Note that our existing integration tests are not that useful here since they mostly check one GCP service at a time while users may use a combination.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r396544685
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
Review comment:
The BOM has dashboard. For 4.2.0, the dashboard URL is https://storage.googleapis.com/cloud-opensource-java-dashboard/com.google.cloud/libraries-bom/4.2.0/artifact_details.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu removed a comment on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
kamilwu removed a comment on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057311
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aromanenko-dev commented on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600751410
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771862
Run Java HadoopFormatIO Performance Test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394420523
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
- google_auth_library_credentials : "com.google.auth:google-auth-library-credentials:$google_auth_version",
- google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
- google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery:1.108.0",
- google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+ google_auth_library_credentials : "com.google.auth:google-auth-library-credentials",
+ google_auth_library_oauth2_http : "com.google.auth:google-auth-library-oauth2-http",
+ google_cloud_bigquery : "com.google.cloud:google-cloud-bigquery",
+ google_cloud_bigquery_storage : "com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core : "com.google.cloud.bigtable:bigtable-client-core:1.13.0",
- google_cloud_core : "com.google.cloud:google-cloud-core:$google_cloud_core_version",
- google_cloud_core_grpc : "com.google.cloud:google-cloud-core-grpc:$google_cloud_core_version",
- google_cloud_datacatalog_v1beta1 : "com.google.cloud:google-cloud-datacatalog:$google_cloud_datacatalog_version",
+ google_cloud_core : "com.google.cloud:google-cloud-core",
+ google_cloud_core_grpc : "com.google.cloud:google-cloud-core-grpc",
+ google_cloud_datacatalog_v1beta1 : "com.google.cloud:google-cloud-datacatalog",
google_cloud_dataflow_java_proto_library_all: "com.google.cloud.dataflow:google-cloud-dataflow-java-proto-library-all:0.5.160304",
Review comment:
This artifact is not in GCP Libraries BOM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771955
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] iemejia commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
iemejia commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600666460
Run BigQueryIO Streaming Performance Test Java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394419843
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin<Project> {
commons_lang3 : "org.apache.commons:commons-lang3:3.9",
commons_math3 : "org.apache.commons:commons-math3:3.6.1",
error_prone_annotations : "com.google.errorprone:error_prone_annotations:2.0.15",
- gax : "com.google.api:gax:$gax_version",
- gax_grpc : "com.google.api:gax-grpc:$gax_version",
+ gax : "com.google.api:gax",
+ gax_grpc : "com.google.api:gax-grpc",
google_api_client : "com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 : "com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 : "com.google.api-client:google-api-client-java6:$google_clients_version",
- google_api_common : "com.google.api:api-common:1.8.1",
+ google_api_common : "com.google.api:api-common",
google_api_services_bigquery : "com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger : "com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager : "com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow : "com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub : "com.google.apis:google-api-services-pubsub:v1-rev20191111-$google_clients_version",
google_api_services_storage : "com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
Review comment:
@elharo FYI, I see these "google-api-services-" Maven artifacts not in our BOM ([list](https://storage.googleapis.com/cloud-opensource-java-dashboard/com.google.cloud/libraries-bom/snapshot/artifact_details.html)). So I keep these dependencies intact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] iemejia commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
iemejia commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600666341
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402662447
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Do you mean comment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] iemejia commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
iemejia commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600666508
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aromanenko-dev commented on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752021
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057763
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] chamikaramj commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402666374
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Yes, please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aromanenko-dev commented on issue #11156: [BEAM-9444] Use
GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600751766
Run BigQueryIO Streaming Performance Test Java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057218
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600933591
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394727429
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Created PR for my suggestion https://github.com/apache/beam/pull/11168
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] suztomo commented on a change in pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
Posted by GitBox <gi...@apache.org>.
suztomo commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402715747
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin<Project> {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
- force project.library.java.values()
+ // Filtering versionless coordinates that depend on BOM
+ def librariesWithVersion = project.library.java.values().findAll { it.split(':').size() > 2 }
+ force librariesWithVersion
Review comment:
Added comment with ticket ID BEAM-9542.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services